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VIDEO EVENT CAPTURE, STORAGE AND PROCESSING 



METHOD AND APPARATUS 



Related Applications 



The application is related to U.S. Provisional Patent Application serial no. 
60/300449, filed June 26, 2001, to which the present application claims priority under 35 
use 119 and which is incorporated herein by reference. 



Background of the Invention 

10 1. Field of the Invention 

The invention relates to a methodology and apparatus adapted to capture still 
images from video sources in real time for the composition of those images. 



2. Description of the Prior Art 

15 There exists a need and desire among a wide range of individuals and 

organizations to capture important, interesting and/or meaningful events for future 
appreciation, viewing and/or other use. Particularly, there is a longstanding desire 
among industry and consumers to obtain still images in the form of silver halide 
photographs or similar hard-^copy images of events of various kinds. It will be 

20 appreciated by those familiar with the photographic art, that there is considerable 

difficulty involved in capturing a precisely defining photograph of an event occurring at a 
single instant in time. The difficulty herein lies both in the photographic composition of 
the subject matter and in the selection of the precise instant at which to take the picture. 
RedHawk Vision Inc. has developed software that generates high quality images 

25 from video source material as set forth in U.S. Patent 4,675,532 and copending U.S. 

Patent applications serial nos. 09/697,017 and 09/427,384, which are incorporated 

1 
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herein by reference. This RedHawk software allows a user to select a precise moment 
captured in video source material, zoom, pan and spatially clip the scene dimensions to 
a desired format size and then generate a high resolution image by combining 
information from the selected video frame with information from temporally nearby video 

ft 

5 frames. The RedHawk software user can 1) choose the precise moment in time; 2) 
spatially frame the subject matter; and 3) determine the length of time (i.e. number of 
video frames) over which the "moment" is to be "taken". Experience with the RedHawk 
software has shown that particularly compelling pictures that portray individual moments 
can be obtained from video of the subject activity much more reliably and easily than 

10 can similar images made by still photography during the event We have also found 
that multiple, different images, derived from a small time segment of a video (e.g. a 
second or less) may each be particularly compelling. Further, several images of 
different subjects taken at the same "moment" are also sometimes useful. The result is 
that the RedHawk software often is used to extract multiple, different images from a 

15 single "moment", all of which images are useful. 

Notwithstanding the considerable usefulness and desirable and advantageous 
characteristics of the current RedHawk software, there remain several limitations and 
disadvantages associated with its use and with the use of frame-grabbers and similar 
technologies, techniques, methods and products available from others for extracting still 

20 images from video. Among the problems and disadvantages of current still extraction 

technologies is the effort required to locate the point within the video source from which 

the image is to be extracted. 

On the one hand, simple frame-grabbers such as the Snappy Hardware Frame 

Grabber allow the user to trigger the "grab" of a video frame in real time, without the 

25 need to re-watch the event on tape to find the "moment" of interest. This offers a 

2 
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significant "woric flow" advantage, however the technique presents all of the timing 
difficulties of real-time, still photography. As a practical matter, the user may end up 
watching an event multiple times by replaying the video or parts thereof, and making 
multiple attempts to precisely **time" capture of the moment of interest, obviating the 
5 work-flow advantage of capturing the event in real-time. 

Alternatively, computer based video editing systems and software such as 
Adobe® Premiere® and specialized stills-from-video systems such as Salient Stills 
Video Focus™ system require that the video of interest be recorded or input in its 
entirety before the moment to be extracted as a still image is extracted. Not 

1 0 withstanding that such systems offer precise control of the moment to be extracted as a 
still image, they all require that the entirety of the video of interest be captured and/or 
input into the system prior to making the still image, thus substantially increasing the 
work flow time and effort over systems that allow capture of an image in real time. It is 
noteworthy that some implementations of the RedHawk software, including those which 

1 5 operate as "plug-in" accessory software to Adobe Premiere exhibit this disadvantage. 

A further limitation of present systems that capture still images from video 
sources is the considerable difficulty that ensues when a user having an extracted still 
image, either in hard copy form, as for instance in the form of a silver halide print, or in 
the form of a digital still image file such as a JPEG file; wishes to obtain a slightly 

20 different image of the same moment. Current systems for extracting still images from 
video do not 'lag" image files with information identifying the source video file or the 
location therein from which the still image was obtained. While still image editing 
software such as Adobe® Photo Shop® allows the addition of meta-data to some still 
image file formats that permit such meta-data to be included, manually, this may still not 
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allow easy location of the video source material and the location within it to allow 
creation of related still images. 

Video files are naturally large. For Instance, an hour of uncompressed video 
having 640 x 480 pixel resolution, 24-bit color resolution and 30 frame per second rate 
5 requires more than 71 billion bytes of storage. The large size of video files makes their 
storage "on-line" problematic In many cases and makes it difficult if not impossible to 
retum to the precise moment in a particular video file or tape from which an image was 
taken to obtain a similar, or related image of the moment of interest. This is particularly 
a problem if a long time has elapsed between the generation of the initial still image and 
1 0 the need or desire for the subsequent image as the video file may no longer be 

available on the computer or, in the case of hardware frame grabbers, the original tape 
may have been lost or discarded. 

Thus, it is an object of the invention to permit the capture of still images from 
video sources in real time with greatly increased precision in the timing and control of 
1 5 the composition of the images. 

It is also an object of the present invention to facilitate the creation of related 
images of a captured moment at a later time. 

It is a further objective of the invention to minimize the storage and transmission 
bandwidth required when obtaining still images from video sources and obtaining hard 
20 copy images there from. 

A yet further object of the invention is to insure availability of adequate storage 
capacity on a users system prior to capturing images to prevent failure to capture an 
image at a critical moment owing to a lack of available storage. 

An additional object of the invention is to inhibit the viewing and/or use of 
25 captured image infonmation by unauthorized persons. 



wo 03/001788 PCTAJS02/19779 



These and other objects of the invention can now be understood by turning to the 
following brief summary. 

Brief Summary of the Invention 
5 The invention is an improvement in a method generating a still image from a 

sequence of video frames comprising the steps of receiving a sequence of video 
frames; storing a temporal sequence of video frames; and copying a predetermined 
number of frames temporally proximate to a capture event to a data structure. The 
predetermined number of frames temporally proximate to the capture event being 
1 0 defined as a cliplet. 

The step of storing a predetermined number of the sequence of video frames 
comprises storing a cliplet of at least one tenth of a second of display time at normal 
frame rates. 

The step of storing a predetermined number of the sequence of video frames 
15 comprises storing an instruction associated with the cliplet relating to a selected frame 
within the cliplet. 

The step of storing an instruction associated with the cliplet comprises storing 
data relating to a selected portion of the selected frame within the cliplet, such as 
storing setting information, like crop dimensions, a crop location, an up-sample ratio, a 
20 sharpness parameter, a persistence parameter, and the number of frames to be 
processed to form a still image, or at least one of such parameters. 

The step of storing a predetermined number of the sequence of video frames 
comprises storing the cliplet in the compressed data fomiat 
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The improvement In the method further comprises the step of processing all 
captured events in the form of cliplets, namely transporting, storing, archiving and 
retrieving video frames or at least one of such processes. 

The step of storing an instruction associated with the cliplet stores a default 
5 instruction and at least one revised instruction relating to the selected frame. 

The improvement of the method further comprises the steps of processing the 
cliplet at a first resolution according to processing instructions determined at the first 
computer, communicating the cliplet and processing instructions together from a first to 
a second computer, processing the cliplet at a second higher resolution according to the 
10 processing instructions detemiined at the first computer to generate a first still image. 

The improvement of the method further comprises the steps of communicating a 
second set of processing instructions between the first and second computer and a 
reference to the corresponding cliplet to the second computer and generating a second 
still image. 

1 5 The improvement of the method further comprises the steps of generating a 

blank cliplet to reserve data space in a buffer. The improvement further comprises the 
step of uniquely identifying the blank cliplet to the client computer and recognizing in a 
server computer the cliplet as uniquely identified with the client computer when the 
cliplet is accessed by the server computer. 

20 The improvement of the method further comprises the steps of encrypting the 

cliplet and communicating it to the server computer in encrypted form, such as by 
storing pseudorandom data in the blank cliplet, and later ovenA/riting the blank cliplet by 
input captured data with the pseudorandom data so that encryption is realized without 
slowing capture of the cliplet. 
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The improvement of the method further comprises the step of exchanging an 
encryption key between the client and server computers at the time the blank cliplet is 
stored into the buffer thereby avoiding exchange of the encryption key after the blank 
cliplet is overwritten and later uploaded into the server computer. The step of 
exchanging the encryption key between the client and server computers occurs during a 
different communication session from when the server computer uploads the encrypted 
cliplet from the client computer. 

The invention is also defined as a method for representing an event, object or 
video data as a still image comprising the steps of providing a temporally proximate 
sequence of video images, defined as a cliplet. The method continues with the steps of 
selecting one of the temporally proximate sequence of video images as a captured 
image; and providing at least one instruction associated with the cliplet to form a 
combined data structure. The data structure, which is the quantum of information 
handled by further evolutions of the cliplet, can be tangibly recorded in a memory circuit, 
a hard disk, a floppy disk, a compact disk or any other storage medium now known or 
later devised. The instruction defines or serves to implement in a computer or data 
processing circuit temporal processing of at least one video image of the cliplet using 
temporally displaced images or data in addition to the captured image to render an 
enhanced still image corresponding to the captured image. 

The step of providing at least one instruction associated with the cliplet provides 
an instruction which also includes spatial processing of at least one video image of the 

■ 

cliplet to render a still image corresponding to the captured image. 

The method further comprises the steps of storing, archiving, transporting, 
processing, or communicating the combined data structure of the cliplet and its 
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associated instruction or instmctions as a single unit of infonnation in a computer 
system. 

The invention further comprises a video viewer, such as a computer and 
computer network controlled by software for performing the foregoing improvements in 
5 the described methodology for generating a still image from a sequence of video 
frames. 

Specifically, the illustrated embodiment of the invention is an improvement to a 
video viewer for use in combination with a display device comprising a computer for 
receiving a sequence of video frames. The display device may be a computer monitor, 

10 a television set or monitor, an LCD screen, or simply an optical site or screen on a 

camera. A buffer stores a temporal sequence of video frames corresponding to what is 
seen in the display device. The computer copies a predetermined number of frames 
which are temporally proximate to a capture event from the buffer to a data structure. A 
"data structure" is information which is stored and which has a predetermined internal 

15 logical structure, which can later be used to interpret, process and exploit the 
information. Thus, "data structure" is also used to refer to a device in which such 
information is stored. The predetermined number of frames are temporally proximate to 
the capture event and are defined as a cliplet. 

While the apparatus and method has or will be described for the sake of 

20 grammatical fluidity with functional explanations, it is to be expressly understood that 

the claims, unless expressly formulated under 35 USC 112, are not to be construed as 

necessarily limited in any way by the construction of "means" or "steps" limitations, but 

are to be accorded the full scope of the meaning and equivalents of the definition 

provided by the claims under the judicial doctrine of equivalents, and in the case where 

25 the claims are expressly formulated under 35 USC 1 12 are to be accorded full statutory 
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equivalents under 35 USC 1 12. The Invention can be better visualized by turning now 
to the following drawings wherein like elements are referenced by like numerals. 

Brief Description of the Drawings 

Fig. 1 is a simplified block diagram of a user computer and network with a server 
in which the viewer of the invention is employed. 

Fig. 2 is a depiction of a screen illustrating the interface of the viewer. 

Fig. 3 is a simplified block diagram of the viewer of the Invention, 

The invention and its various embodiments can now be better understood by 
turning to the following detailed description of the preferred embodiments which are 
presented as illustrated examples of the invention defined in the claims. It is expressly 
understood that the Invention as defined by the claims may be broader than the 
illustrated embodiments described below. 

Detailed Description of the Preferred Embodiments 

Generating a still image from a sequence of video frames comprises the steps of 
receiving a sequence of video frames and storing a predetermined number of the 
sequence of video frames temporally adjacent to an user initiated capture event, the 
predetermined number of frames being defined as a cliplet. Video capture is performed 
using cliplets, which are short video segments, preferably in compressed data format. A 
crop-and-process instruction is added to the cliplet. All processing, handling or storage 
of the data is cliplet based. Editing instructions are stored with the cliplet allowing 
editing to be performed by a user at one resolution and then re-performed at a higher 
resolution at a server where it is printed. Modified reprints of an image can be obtained 
without resending the images and adequate storage for the cliplets can be assured. 



* 
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Encryption of the cllplets during transmission and storage is facilitated by cliplet-based 
processing. 

Snap-and-Go 

A first aspect of the invention is referred to as "Snap-and-Go". As shown in Fig. 
5 1 this aspect of the invention comprises a video-viewer 10, implemented preferably in 
software and operated on a computer 12 to display video from a file 18 within computer 
12, from a digitizer source 14 connected to the computer 12, or from a video file 
streamed over an internet connection 16 to computer 12 or fed to computer 12 by some 
other means. 

1 0 Viewer 1 0 is similar to the Apple Quicktime® video player to the extent that, as 

shown in the screen depiction of Fig. 2, it provides buttons for the features of window 
resizability 26, play 20. stop 22, and positioning 24 (if playing from a file or such) and in 
addition, includes a "capture" button 28 (or similar functionality such as a "hot-key" or 
infrared link triggering means). Activation of the "capture" button 28 causes a short 

15 segment of the video about the time of the "capture" activation to be saved. Preferably, 
the section of video saved both precedes and follows the instant of "capture" activation 
by a second or so. Viewer 10 of this Invention as shown in Fig. 3 advantageously 
includes a buffer 30 that stores a second or so of the video most recently displayed so 
that it can be saved upon activation of the "capture" button 28. 

20 

Adding "Crofhand-ProcesS'lnstruction" to Cliplet 

When a short video segment 32 (referred to as a "cliplet") is saved upon 
"capture" by activation of button 28, it is saved together with a short instruction 34 or 

10 
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data identifying a particular frame within the cliplet 32 and describing a section within 
the frame from which a still image is to be extracted. Data 34 is alternatively termed a 
"crop-and-process-instruction". I\/lost preferably, this "crop-and-process-instruction" 34 
includes the frame number within the cliplet 32 for the desired picture and also setting 
5 information such as crop dimensions, crop location, up-sample ratio, sharpness 
parameter, persistence parameter, number of frames, etc. needed by the RedHawk 
software stored in computer 12 to extract a high resolution image from the video cliplet 
32. Such RedHawk software is called Paparazzi Pics™. An example of an instruction 
is shown in Table I. 

10 

Table I 



Instruction Field 

* 


Parameter Value 


Frame No. 


17 


Framing 


(7,20) (502, 350) 


Number of frames 


4.5 


Sharpness 


8 


Persistence 


4 


Print Size 


5x7 



In this illustration the capture event has designated frame #17, for example, from 
a temporal sequence of video frames as the desired image from which a still is to be 
15 derived. The framing of the image designates the pixel location of the top left and the 
bottom right comer of the frame. In this way a portrait or landscape format for the still 
image can be specified. The number of frames in the cliplet 32 can be specified. Since 

11 
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video images are comprised of interleaved scans of even and odd lines, it is possible to 
specify half frames, which is advantageous when blur of a moving image needs to be 
removed or clarified in the still. Sharpness refers to a parameter used in the algorithm 
by which still is composed of the multiple frames, which the user can select and view by 
5 trial and error until the desire effect is achieved. Persistence refers to a weighting given 
to frames temporally proximate to the captured frame. For example, "persistence" may 
a number from 1 to 10 which provides equal weight to all frames for a value of 10, 
decreasing with a range of stepped weights given to temporally proximate frames for 
values of 9 to 2, down to no weight to any frame other than the capture frame for a 
10 value of 1 . Print size designates what size of print the user desires, e.g. 4 x 6, 5 x 7, 8 x 
10 and the lil<e. It is to be understood that the instruction 34 may have many different 
formats and include more or less information than in the example and may include other 
types information relating to processing or disposition of the still to be derived from 
cliplet 32. 

15 Upon "capture" activation by button 28, the "crop-and-process-instruction" 34 is 

set to nominal or default values. For instance, the frame number may be selected as 
that frame displayed at the instant of "capture" activation by button 28; the crop region 
and other parameters set to the default values of the current RedHawk software, 
Paparazzi Pics™. It will be understood by those familiar with the art that the "crop-and- 

20 process-instruction" 34 may also or alternatively contain information to direct image 

extraction processes of other software routines, rather than specifically the RedHawk 

software. For instance, the "crop-and-process-instruction" might contain information 

needed to perform a single frame grab, or to direct another multi-frame image extraction 

program such as VIDEO FOCUS® manufactured by Salient Stills Company of Boston, 

25 Massachusetts or any other photo processing software now known or later devised. 

12 
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Still further, more than one instruction 34 can be associated with cliplet 32. For 
example, a first instruction 34 might provide instructions for a 8 x 10 overall shot of the 
subject, while a second instaiction 34 associated with cliplet 32 may provide instructions 
to crop the photo, enlarge and enhance one section of the overall shot, such as a close 
of one person in a group photo. A single cliplet 32 will thus be processed with multiple 
instructions as a unit to produce a multiple of corresponding individualized stills. 

Cliplet comprising Compressed Video 

Another aspect of the invention is the storage of video frames of each cliplet in 
compressed format. It is widely appreciated that a sequence of video frames often 
comprises a sequence of closely related images. Several image compression schemes 
are currently in use that take advantage of this fact to achieve high degrees of data 
compression for video files. In fact, video files from many digital video cameras are, in 
fact, compressed video files, as frequently are video files streamed over the Internet 
and files processed by many video digitizer boards that process analog video into digital 
files for display and manipulation by computers. Viewer 10 typically deals with 
compressed video files. Particularly, the viewer software retrieves video frames in 
compressed fomri (for instance by using a Quicktime call), then directs the compressed 
frame data to a decompression routine that is part of the computer's operating system 
and associated support software such as Microsoft MEDIA PLAYER® manufactured by 
Microsoft Inc. of Bellevue, Washington or QUICKTIME® manufactured by Apple 
Computer Inc. of Cupertino, California. By causing the viewer to store the cliplet in the 
form of compressed video, that is by maintaining the previously described video buffer 

30 in compressed frame format as delivered from the source and storing that as the 

« 

cliplet video sequence, the size of each cliplet file is greatly reduced. It is possible, in 

13 
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the case of uncompressed video source material, for viewer 10 to compress the video 
frames in its video buffer 32 by any of several available compression algorithms like 
INDEO® manufactured by Intel Corp. of Santa Clara. Califomia. SORENSON® 
manufactured by Sorenson Media of Salt Lake City, Utah, or by any other compression 
5 algorithm now known or later devised. In this way, the size of cliplets obtained even 
from uncompressed video files can be greatly reduced. 

Cliplet + process Instruction Smaller than RedHawk High Resolution File -Without 
Resolution Loss 

10 A further aspect of the invention is the transport, storage, archiving and retrieval 

of information as cliplets, including the related crop-and-process-instruction 34, rather 
than as high resolution still image files in TIFF or JPEG formats, for instance. This 
offers several advantages over the storage, archiving and retrieval of still image 
Information as is currently done for still images over the internet by SHUTTERFLY of 

15 Redwood City, California, KODAK of Rochester, New York, PHOTO WORKS of Seattle, 
Washington, and several others. Among the advantages offered by this aspect of the 
invention is reduced total file size without loss of image quality. Specifically, consider 
that a RedHawk produced high resolution image made at an up-sample ratio of 3x 
contains nine times as many pixels as a single frame of video from which it is derived. 

20 Further consider that this RedHawk high resolution image can often be made from four 

or fewer frames of video, so if the associated cliplet contains even 8 frames of the video 

(twice those used to create the still image) the total number of pixels represented in the 

8 frame video cliplet is fewer than the number of pixels in the RedHawk high resolution 

still image. It will be apparent to those skilled in the art that if the cliplet contains 

25 compressed video data from the original source file, the high resolution source image 

14 
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produced by RedHawk processing will be precisely the same whether the RedHawk 
processing is applied to the data before or after transport, storage, retrieval, etc. of the 
cliplet. This is not the case, however, if the RedHawk high resolution image itself is 
compressed (by JPEG for instance) and then transported, stored, retrieved, etc. 
5 because additional information is lost in the compression of the high resolution JPEG or 
similar file. 

Editing in Both Time and Space 

A further additional aspect of the invention is the use of a precision video to still 

10 image processing tool like the RedHawk software or Salient Stills' VIDEO FOCUS 
system to extract varied Images from a cliplet. When viewer 10 is used to "capture" 
cliplets of video and assign a "default" crop-and-process-instruction 34 thereto, a 
difficulty arises in obtaining a precisely timed image that is similar to that described 
eariier for still image photography. By using the RedHawk precision still image from 

15 video software, or similar to select more precisely the exact instant to capture from 
within a cliplet, precision timing is achieved while the user continues most of the work- 
flow advantage obtained when "capturing" images during real-time video play. 
Advantageously, the RedHawk software is configured so that its crop-and-process 
parameters are set to those associated with the cliplet when the cliplet is loaded. 

20 Further, the RedHawk software may be advantageously configured to store a different 
crop-and-process-instruction 34 back to the cliplet file representing the user's revised 
selection of parameters. This may be done either by replacing the default crop-and- 
process-instruction 34 with the revised parameters or by appending an additional crop- 
and-process-instruction 34 to the cliplet file. 

25 
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Pmcessing at One Resolution then Sending and Printing at a Second Resolution 

A further aspect of the invention involves the "capture" of a cliplet and generation 
and/or display of the RedHawl< (or similar) high resolution image and optionally editing 
in time and space (i.e. editing of the crop-and-process-instruction 34) at a first location 
5 36; then transmission of the cliplet file to a second computer 12' at a second location 38 
and generating a second RedHawk (or similar) image based on the attached crop-and- 
processnnstruction 34. In the case where a high resolution image is to be produced at 
second location 38, as for example when ordering a silver halide print, a different up- 
sample ratio may advantageously used when the RedHawk algorithm is applied at 

10 second location 38. This aspect of the invention is particularly advantageous when a 
user is processing cliplets on a computer 12 and displaying the resulting images on a 
limited resolution computer screen 40 and then sends the cliplet file to a remote location 
38 where a high quality (e.g. silver halide) hard copy image is to be produced on a 
photographic quality printer 42. Specifically, this arrangement allows 1) fast RedHawk 

15 processing with a low up-sample resolution at the first computer 12 since no resolution 
beyond what can be displayed on a low resolution computer display 40 may be 
required; 2) low bandwidth and storage requirements for transporting the cliplet to 
second location 38; and 3) production of a very high resolution image on printer 42 at 
second location 38 that is faithful to the first lower resolution RedHawk image. 

20 

Obtaining Modified Re-Prints \Mtliout Re-sending Images 

A further aspect of the invention involves capture of a cliplet at a first location 36, 
optionally editing the crop-and-process-instruction 34 so as to define a first desired 
image; sending the cliplet with the crop-and-process-instruction 34 to second location 
25 38 where the cliplet is stored; again editing the crop-and-processes-instruction 34 at first 
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location 36 so as to produce a second desired image that differs in timing, cropping or 
other RedHawk processing parameter; and then transmitting the second crop-and- 
process-instruction 34 together with a reference to the originally transmitted cliplet to 
second location 38 to generate a second image at second location 38. This aspect of 
5 the invention offers very much reduced transmission time compared with sending the 
two high resolution images, or sending the cliplet a second time with the second crop- 
and-process-instruction 34. 

Insuring Adequate Storage Availability when Capturing Cliplets 

10 Another aspect of the invention is the generation and storage of a "blank" cliplet 

storage file of size adequate to store a predetermined number of cliplets. By causing 
such a cliplet storage file, filled with zeros or with random data, to be written to a 
computer's reusable storage (e.g. hard disk drive), storage space is "reserved" that is 
adequate to store the predetermined number of images. When a cliplet is "captured", 

15 as described above, the cliplet information is written over information in the existing 

cliplet storage file. By this means, the user is assured that the predetermined number of 
cliplets can be captured and stored without encountering a "disk-full" condition. 
Advantageously, the cliplet storage file is named or otherwise tagged (for instance with 
user ID, date, time and machine GUID) at the time it is created. 

20 A further advantageous feature is implemented when the user communicates 

with a central server 44 in connection with generating a cliplet storage file 46 such that a 
unique identifier may be shared between the server 44 and the cliplet storage file 46 
such that when and if the cliplet storage file 46 is sent to a networked server 44, it may 
be recognized as belonging to a particular user and filed on server 44 accordingly. 

25 
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Insuring Security of Cliplet during Transmission and Storage 

An advantageous aspect of the invention is the use of encryption when storing 
cliplets and/or crop-and-process-instructions 34 to the cliplet storage file 46. This 
aspect of the invention is useful as a way to prevent viewing or use of either the final 
image or the cliplets by unauthorized persons. 

A particularly advantageous implementation of this cliplet encryption feature 
results when a cliplet storage file 46 is generated in connection with a server 44. If the 
user computer 12 and the server 44 exchange both an ID and encryption key and the 
encrypted file is then sent to the server 44, the cliplet data is protected and the server 
44 can have prior knowledge of how to decrypt the file 46. An especially preferred 
implementation of this feature causes the user computer 12 to store a "blank" cliplet 
storage file 46 that is filled with pseudo-random number sequences generated from a 
key exchanged with the server 44. Cliplet data to be stored into the cliplet file 46 is 
combined with the pseudo-random sequence and over written into the file 46. This 
implementation thus combines both storage reservation and encryption, and also 
simplifies and therefore speeds up the encryption process at the time the cliplet 32 is 
stored, since the pseudo-random sequence is generated prior to cliplet capture. 

Since the user computer 12 and the server 44 exchange the encryption key at 
the time the cliplet storage file 46 is initialized, there needs to be no exchange of keys at 
the time the file 46 is uploaded to the server 44. Advantageously, the exchange of keys 
between the server 44 and the user's computer 12 is done in a secure (i.e. https) 
network session, and further the encrypted cliplet storage file 46 is transmitted during a 
separate session, preferably at a later time. This makes It is quite difficult for a would- 
be eaves dropper to recover both the cliplet storage file 46 and the encryption key. 
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Many alterations and modifications may be made by those having ordinary skill in 
the art without departing from the spirit and scope of the Invention. Therefore, it must 
be understood that the illustrated embodiment has been set forth only for the purposes 
of example and that it should not be taken as limiting the invention as defined by the 
5 following claims. For example, notwithstanding the fact that the elements of a claim are 
set forth below in a certain combination, it must be expressly understood that the 
invention includes other combinations of fewer, more or different elements, which are 
disclosed in above even when not initially claimed in such combinations. 

The words used in this specification to describe the invention and its various 

1 0 embodiments are to be understood not only in the sense of their commonly defined 
meanings, but to include by special definition in this specification structure, material or 
acts beyond the scope of the commonly defined meanings. Thus if an element can be 
understood in the context of this specification as including more than one meaning, then 
its use in a claim must be understood as being generic to all possible meanings 

1 5 supported by the specification and by the word itself. 

The definifions of the words or elements of the following claims are, therefore, 
defined in this specification to include not only the combination of elements which are 
literally set forth, but all equivalent structure, material or acts for performing substantially 
the same function in substantially the same way to obtain substantially the same result. 

20 In this sense it is therefore contemplated that an equivalent substitution of two or more 
elements may be made for any one of the elements in the claims below or that a single 
element may be substituted for two or more elements in a claim. Although elements 
may be described above as acting in certain combinations and even initially claimed as 
such, it is to be expressly understood that one or more elements from a claimed 

25 combination can in some cases be excised from the combination and that the claimed 
combination may be directed to a subcombination or variation of a subcombination. 

Insubstantial changes from the claimed subject matter as viewed by a person 
with ordinary skill in the art, now known or later devised, are expressly contemplated as 
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being equivalently within the scope of the claims. Therefore, obvious substitutions now 
or later known to one with ordinary skill in the art are defined to be within the scope of 
the defined elements. 

The claims are thus to be understood to include what is specifically illustrated 
5 and described above, what is conceptionally equivalent, what can be obviously 

substituted and also what essentially incorporates the essential idea of the invention. 
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I claim: 

1 1 . An improvement to a video viewer for use in combination with a display device 

2 comprising: 

3 a computer for receiving a sequence of video frames; and 

4 a buffer for storing a temporal sequence of video frames corresponding to what is 

5 seen in the display device, the computer copying a predetermined number of frames 

6 temporally proximate to a capture event from the buffer to a data structure, the 

7 predetermined number of frames temporally proximate to the capture event being 

8 defined as a cliplet. 

1 2. The improvement of claim 1 where the computer stores a cliplet in the buffer 

2 which cliplet comprises at least one tenth of a second of display time at normal frame 

3 rates. 

1 3. The improvement of claim 1 where computer stores an instruction in the buffer 

2 associated with the cliplet relating to a selected frame within the cliplet. 

1 4. The improvement of claim 3 where the instruction stored by the computer in the 

2 buffer associated with the cliplet further relates to a selected portion of the selected 

3 frame within the cliplet. 
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1 5. The Improvement of claim 4 where the Instruction stored by the computer in the 

2 buffer associated with the cliplet further relating to a selected portion of the selected 

3 frame within the cliplet includes setting Information. 

1 6. The improvement of claim 5 where the setting Information relates to parameters 

2 such as crop dimensions, a crop location, an up-sample ratio, a sharpness parameter, a 

3 persistence parameter, and the number of frames to be processed to form a still image, 

4 or at least one of such parameters. 

1 7. The improvement of claim 1 where the cliplet Is communicated to the computer in 

2 a compressed data fonmat and where the computer stores the cliplet In the buffer in the 

3 compressed data format. 

1 8. The Improvement of claim 1 where the computer processes all video frames in 

2 the form of clipiets. 

1 9. The improvement of claim 8 where the computer performs the processes of 

2 transporting, storing, archiving and retrieving clipiets or at least one of such processes. 

1 10. The improvement of claim 4 where the Instruction stored by the computer In the 

2 buffer associated with the cliplet corresponds to a default instruction, and further 

3 includes at least one revised instruction relating to the selected frame. 

22 
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1 11. The improvement of claim 1 where the computer is a first computer which 

2 processes the cliplet at a first resolution, and further comprising a second computer 

3 communicated to the first computer, the second computer processing the cliplet at a 

4 second higher resolution according to processing instmctions determined at the first 

5 computer to generate a first still image, the cliplet being communicated between the first 

6 and second computers together with the processing instructions. 

1 12. The improvement of claim 1 1 where the first computer is communicated with the 

2 second computer to transport a second set of processing instructions and a reference to 

3 the corresponding cliplet to the second computer to generate a second still image. 

1 13. The improvement of claim 1 where the computer generates a blank cliplet in the 

2 buffer to reserve data space therein. 

1 14. The improvement of claim 1 3 wherein the computer is a client computer and 

2 wherein a blank cliplet is uniquely identified to the client computer and further 

3 comprising a server computer communicated to the client computer, and wherein the 

4 server computer recognizes the cliplet as uniquely identified with the client computer 

5 when the cliplet is accessed by the server computer. 

1 15. The improvement of claim 14 wherein the cliplet is encrypted in the buffer and 

2 communicated to the server computer In encrypted form. 
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16. The improvement of claim 15 where the client computer stores the blank cliplet 
as pseudorandom data and later when the cliplet is ovenA^ritten by the client computer 
onto the blank cliplet, it is combined with the pseudorandom data whereby encryption is 
realized without slowing capture of the cliplet. 

17. The improvement of claim 15 where the client and server computers exchange 
an encryption key at the time the blank cliplet is stored into the buffer thereby avoiding 
exchange of the encryption key after the blank cliplet is ovenA^ritten and later uploaded 
into the server computer. 

18. The improvement of claim 17 where the encryption key is exchanged between 
the client and server computers at a different communication session from when the 
server computer uploads the encrypted cliplet from the client computer. 

19. An improvement in a method generating a still image from a sequence of video 
frames comprising: 

receiving a sequence of video frames; and 
storing a temporal sequence of video frames; and 

copying a predetermined number of frames temporally proximate to a capture 
event to a data structure, the predetermined number of frames temporally proximate to 
the capture event being defined as a cliplet. 
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20. The improvement of claim 19 where storing a predetermined number of the 
sequence of video frames comprises storing the cliplet with at least one tenth of a 
second of display time at normal frame rates. 

21 . The Improvement of claim 19 storing a predetermined number of the sequence of 
video frames comprises storing an Instruction associated with the cliplet relating to a 
selected frame within the cliplet. 

22. The improvement of claim 21 where storing an instruction associated with the 
cliplet comprises storing data relating to a selected portion of the selected frame within 
the cliplet. 

23. The improvement of claim 22 where storing data relating to a selected portion of 
the selected frame within the cliplet comprises storing setting information. 

24. The improvement of claim 23 where storing setting information comprises storing 
parameters such as crop dimensions, a crop location, an up-sample ratio, a sharpness 
parameter, a persistence parameter, and the number of frames to be processed to form 
a still image, or at least one of such parameters. 

25. The improvement of claim 19 where storing a predetermined number of the 
sequence of video frames comprises storing the cliplet in the compressed data format. 
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1 26. The improvement of claim 19 further comprising processing all video frames In 

2 the form of cliplets. 

1 27. The improvement of claim 26 where processing all video frames in the form of 

2 cliplets comprises transporting, storing, archiving and retrieving video frames or at least 

3 one of such processes. 

1 28. The improvement of claim 22 where storing an instruction associated with the 

2 cliplet stores a default instruction and at least one revised Instruction relating to the 

3 selected frame. 

1 29. The improvement of claim 19 further comprising processing the cliplet at a first 

2 resolution according to processing instructions determined at the first computer, 

3 communicating the cliplet and processing instructions together from a first to a second 

4 computer, processing the cliplet at a second higher resolution according to the 

5 processing instructions determined at the first computer to generate a first still image. 

1 30. The improvement of claim 29 further comprising communicating a second set of 

2 processing instructions between the first and second computer and a reference to the 

3 corresponding cliplet to the second computer and generating a second still image. 
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31. The improvement of claim 19 further comprising generating a blank cliplet to 
reserve data space in a buffer. 
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32. The improvement of claim 31 further comprising uniquely identifying the blank 
cliplet to the client computer and recognizing in a server computer the cliplet as uniquely 
identified with the client computer when the cliplet is accessed by the server computer. 

33. The improvement of claim 32 further comprising encrypting the cliplet and 
communicating it to the server computer in encrypted form. 

34. The improvement of claim 33 further comprising storing pseudorandom data in 
the blank cliplet, later ovenwriting the blank cliplet by Input captured data with the 
pseudorandom data so that encryption is realized without slowing capture of the cliplet. 

35- The improvement of claim 33 further comprising exchanging an encryption key 
between the client and server computers at the time the blank cliplet is stored into the 
buffer thereby avoiding exchange of the encryption key after the blank cliplet is 
overwritten and later uploaded into the server computer. 

a • 

36. The improvement of claim 35 further comprising exchanging the encryption key 
between the client and server computers during a different communication session from 
when the server computer uploads the encrypted cliplet from the client computer. 

37. A method for representing an event as a still image comprising: 

providing a temporally proximate sequence of video images, defined as a cliplet; 
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3 selecting one of the temporally proximate sequence of video images as a 

4 captured image; and 

5 providing at least one Instruction associated with the cliplet to form a combined 

6 data structure, which instruction defines temporal processing of more than one video 

7 image of the cliplet to render a still image corresponding to the captured image. 

1 38. The method of claim 37 where providing at least one instruction associated with 

2 the cliplet provides an instruction which also includes spatial processing of at least one 

3 video Image of the cliplet to render a still image corresponding to the captured image. 

4 39. The method of claim 37 further comprising storing, archiving, transporting, 

5 processing, or communicating the combined data structure of the cliplet and its 

6 associated instruction as a single unit of information in a computer system. 
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